The first document is The Jewish War by Josephus who, as a Jew, did not have a dog in this fight. Writing is the last quarter of the first century, he referred to the martyrdom of James, whom he called the brother of Jesus, previously mentioned in the book.
Some time around AD 150 the Christian author, Hegesippus travelled around the east gathering oral traditions about the bishops of the churches. He also related the martyrdom of the leader of the Jerusalem church, James the brother of Jesus, and how he was replaced by Simeon, who was the son of St. Joseph's brother, Cleophas. He also referred to the grandsons of another of Jesus' brothers, Judas (almost certainly the author of the Epistle of Jude).
The first support for the perpetual virginity of Mary appeared some time in the middle of the second century, or perhaps a little later, in the form of an apocryphal work known, among other things, as the Protoevangelium ("first gospel") of James, which purports of relate the story of Jesus' birth, but was really concerned with glorifying His mother. You can read it here.
It begins with an aged but childless couple, Joachim and Anne praying for a child, and Anne vowing that the child will be committed to the temple. Angels are sent to both of them to announce the conception of the child. They call her Mary, and when she is three years old, she is brought to the temple, where she must have suffered a lonely childhood, although she is fed by an angel. However, when she turns twelve, the priests get concerned that, when she has her first period, the temple will be defiled. An angel then tells them to marry her off to a pious widower, and the lot falls to Joseph. Being elderly, with children of his own, he initially rejects the role, but is eventually persuaded to comply. He then leaves her at home while he goes off to construct his buildings. Meanwhile, Mary is chosen to spin the purple and scarlet cloth for the curtain of the temple. Gabriel comes to announce the conception of Jesus, and she goes to stay with Elizabeth. By now, it is said, she was aged sixteen, which means Joseph must have been away four years. When he returns, he finds her already six months pregnant. Although God assures him in a dream of the truth, this is not sufficient to satisfy the priests, until she has undergone the ordeal demanded in Numbers 5:11-31. After that, there is the story of the journey to Bethlehem, of Jesus’ birth in a cave outside the town, and how the midwives confirmed that Mary’s hymen was unbroken both before and after the birth.
Now the interesting thing is that the Protoevangelium has always been known to be a work of fiction. It never came close to being treated as scripture. It didn't even rise to the level of being a disputed book, or even one of those regarded as useful to read but not to establish doctrine. However, in the following centuries it became extremely influential, and both Joachim and Anne are saints on the calendar of both the Roman and Eastern Orthodox churches. Furthermore, you will still hear some of their scholars claim that the Virgin Mary was some sort of female temple worker.
Early in the third century, the prolific Christian writer, Tertullian tackled the heresy of Docetism - the idea that Christ’s body was illusionary - in a tract entitled, On the Flesh of Christ. Of course, he insisted, Christ’s body was solid and human; he had a mother and brothers. Although not stated, the argument would have been void if the brothers were not the biological sons of his biological mother. Even Jerome (see Part 4) accepted that this was Tertullian’s position. In the 230s an even more prolific Christian author, Origen wrote in his Commentary on Matthew 10:17
Now some, using a tradition from the Gospel with the title, According to Peter or the Book of James as their starting point, say that the brothers of Jesus were the sons of Joseph by a wife prior to Mary. Those who say this wish to preserve the dignity of Mary in virginity until the end.
Origen thought this was reasonable, and he repeated it emphatically in his Homily on Luke 7:4. Nevertheless, the fact that "some" held the view implies that some others held the opposite view. Probably they were the majority first, because people don't say "some" when they mean "most", and secondly, because the Protoevangelium of James, not being scripture, was unlikely to have been widely read by the rank and file at that time. It is no doubt not a coincidence that Origen was not married but Tertullian was.
On the other hand, Victorinus of Pettau, who was martyred about 303 or 304, is said to have considered the brothers of Christ to be the biological sons of Mary. However, his writings have been lost.
And that, essentially, is all we have up till about AD 300: an apocryphal work of fiction, a theologian who read it and thought it reasonable, and a number of others who appear to adopt the natural meaning of the biblical text. Some of these are controversial because the references were brief, but that is surely the point: for the first 270 years of Christianity the issue was a matter of indifference to the church. Few writers addressed it. It was not considered anything to get passionate about. Probably anybody who bothered to think about it took the obvious view that Jesus’ siblings were the children of Mary and Joseph. There certainly wasn’t any additional information.
However, beginning in 300 we begin to see writer after writer expounding the doctrine of the perpetual virginity. What changed? Simply, the introduction of the monastic movement, begun in the first decades of the fourth century. In the declining era of the Roman Empire, when the world seemed to be falling apart, the idea of escaping from the evils of the world in such a manner spread like wildfire. And it placed a high premium on the celibate life. This movement really did believe that lifelong virginity was holier than that of a virtuous married person.
Thus, St. Gregory of Nyssa (c 335 - c 394) always regretted the fact that, as a widower, he could not share the "glory" of lifelong virginity. He felt like the confectioners and attendants who prepare a rich banquet but cannot partake of it. What a way to remember one's relationship with one's late wife! But you can easily see how, with this sort of attitude doing the rounds, people would easily believe that a holy couple would choose the live in a sexless marriage.
The Church of Rome in particular claims to base its doctrine on sacred tradition as well as the Bible. However, tradition is only as strong as its weakest link. The church cannot get a new tradition, and if something was just a possibility in the second and third centuries, it cannot be an established doctrine by the fourth - especially in the absence of evidence.
